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Abstract 


I n 2009, some Spanish regions implemented the Program School 2.0 with 
the purpose of introducing digital methodologies at schools. The aim 
of this paper is to analyse which part of the variation in reading scores is 
due to this program. For this purpose, we use data from the Program for 
International Student Assessment (PISA 2009 and 2012) for 15-year old 
students attending public schools. We estimate a difference-in-difference 
model and observe that the net effect derived from an increase in the 
provision of Information and Communications Technology (ICT) at schools 
has been positive, although small, in participant regions. However, elapsed 
time since the onset of the program has not equally affected repeater and 
non-repeater students. Finally, only a moderate use (1-2 times/week) of ICT 
for doing homework has a positive effect over reading scores. 


Keywords: reading, PISA, ICT, Spain. 


1. Introduction 

The analysis of the implementation of ICT in schools and high schools has 
sparked debate during the last decade. Some studies have appreciated a 
substantial improvement of students’ achievement as a result of the introduction 
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of ICT. Machin, McNally, and Silva (2007) used an instrumental variables 
approach to control for a potential endogeneity problem of the use of ICT, and 
concluded that the increase in computer investment had improved academic 
results in Elementary education. In the same line, Banerjee, Cole, Duflo, and 
Linden (2004) for India, Barrow, Nmarkman, and Rouse (2009) for the United 
States and Carrillo, Onofa, and Ponce (2010) for Canada, ascertained a positive 
influence of ICT over academic results. 

However, other analyses have found an insignificant or even negative relationship 
between both variables. Golsbee and Guryan (2002) concluded that a program 
implemented in the United States aimed at increasing the computer-to-student 
ratio had not had any significant effect over students’ achievement. For Israel, 
Angrist and Lavy (2002) observed a negative effect of ICT over Mathematics 
scores for 4th grade students. Similarly, Leuven, Lindahl, Oosterbeek, and 
Webbink (2004) concluded that the increase of computer-to-student ratio in 
Dutch schools had led to worse Language and Mathematics results. 


2. The program school 2.0 

In July 2009, the Spanish Ministry of Education approved the development of 
the Program School 2.0, whose objectives were: provide each student with a 
notebook or digital pad, transform all classrooms into digital classrooms, offer 
instruction to teachers and prepare new digital contents. 

The program was implemented in 5th and 6th grade of Elementary Education 
and 1st and 2nd grade of High School, but only in public centers. Participation 
in the Program was not homogeneous across Communities, and the following 
classification can be established (see CEAPA, 2010): 

• Communities that applied the Program in all centers, denoted ‘Total 
Participants’ (TP): Andalucia, Aragon, Cantabria, Castilla La Mancha, 
Castilla-Leon, Cataluna, Extremadura, Galicia, Navarra, Pais Vasco, 
Rioja, Ceuta and Melilla. 
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• Communities that applied the Program in a fraction of centers denoted 
as ‘Partial Participants’ (PP) Asturias, Baleares and Canarias. 

• Communities that did not implemented the Program, denoted as ‘Non- 
Participants’ (NP): Madrid, Murcia and Comunidad Valenciana. 


3. Data 

Data come from PISA survey carried out by the Organisation for Economic 
Cooperation and Development (OECD) every three years to assess the 
competencies of 15-year-old students in reading, mathematics and science. This 
paper is focused on students with level ISCED-2A 2 attending public centers. 
We have a sample of 15,375 observations for the general module and 5,579 
observations for the Computer Based Assessment (CB A) module. 

To assess the success degree of the Program School 2.0, it is necessary to 
compare reading scores in 2012 with pre-implementation scores. We incorporate 
11,049 observations from PISA 2009 and 1,897 from PISA-Electronic Reading 
Assessment (ERA). 

Table 1 shows reading scores in 2009 and 2012 by type of participation. 
For non-repeater students, there is no significant difference among the three 
types of Communities, neither in 2009 nor in 2012. In the modules ERA 
(2009) and CBA (2012), we appreciate that NP and TP attain higher scores 
than PP. For 1-year repeaters, mean score for TP was higher than for NP in 
2009, but quite the opposite happens in the module CBA (2012). 

For 2-year repeater students, the mean score for NP was higher than for PP 
according to PISA-ERA (2009), but no significant differences are observed 
in electronic reading achievement in 2012. 


2. International Standard Classification of Education; 2A: Secondary School Intermediate Level 
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Table 1. Descriptive statistics for reading scores 



Has participated in School 2.0? 

Test for equal means 


No (1) 

Totally (2) 

Partially (3) 

< 

C/3 

'kO 

(1 )vs (3) 

(2) vs (3) 

PISA (2009). General Module 

Total 

436.80 

446.59 

450.79 

0.0613 

0.0071 

0.1266 

No rep 

490.94 

489.83 

495.94 

0.6274 

0.5828 

0.1888 

1-year rep. 

397.22 

410.66 

416.00 

0.0055 

0.0082 

0.6360 

2-year rep. 

347.26 

342.49 

353.60 

0.1835 

0.1762 

0.8312 

PISA (2009). ERA 

Total 

488.82 

481.82 

438.28 

0.7392 

0.0000 

0.0000 

No rep 

522.84 

514.80 

487.51 

0.7627 

0.0000 

0.0000 

1-year rep. 

447.51 

441.83 

409.91 

0.2631 

0.1692 

0.0310 

2-year rep. 

416.88 

410.65 

383.09 

0.7372 

0.0064 

0.0157 

PISA (2012). General Module 

Total 

477.14 

480.21 

457.54 

0.0116 

0.6465 

0.0596 

No rep 

515.26 

513.98 

502.00 

0.4394 

0.1898 

0.3533 

1-year rep. 

440.54 

431.44 

421.68 

0.2044 

0.3079 

0.1736 

2-year rep. 

390.63 

381.16 

375.27 

0.9505 

0.6076 

0.5060 

PISA (2012). CBA 

Total 

470.77 

477.89 

457.48 

0.1478 

0.1107 

0.0066 

No rep 

507.35 

512.51 

490.67 

0.8205 

0.0373 

0.0252 

1-year rep. 

437.98 

423.59 

420.74 

0.0001 

0.0861 

0.5544 

2-year rep. 

379.57 

380.60 

378.91 

0.8595 

0.6036 

0.8675 


Table 2 shows the degree of use of ICT at schools and students’ households 
according to the type of participation in the Program School 2.0. In 2009, all 
Communities exhibited similar levels of technological equipment at schools 
(0. 15-0. 16). In 2012, the highest ratio of computer-per-student corresponds to TP 
Communities (0.65). Regarding the provision of technological equipment, there 
has been a higher investment in PCs in PP Communities (69%) in comparison 
with notebooks in TP Communities (31%). 


Nearly 20% of students belonging to TP or PP Communities have reported that 
they use ICT for ‘looking for information’ at school ‘almost every day’ or ‘every 
day’, as opposed to only 12% in NP Communities. In the context of using ICT 
for ‘practice/drilling’ or ‘doing homework at school’, the percentage is higher 
in PP Communities (14% and 11%) as opposed to TP Communities (9%). 
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Finally, around 12% of students of TP or PP have reported to use ICT to do their 
homework (at home) ‘almost every day’ or ‘every day’. 


Table 2. Implementation of ICT atschoolsandstudents’households;PISA(2012) 



Autonomous Communities 

Total 

participants 

Partial 

participants 

No 

participants 

Ratio computers-per-student 2009 

0.15 

0.15 

0.16 

Ratio computers-per-student 2012 

0.65 

0.63 

0.57 

At classroom, the student has PC (%) 

56.6 

69.14 

61.43 

At classroom, the student has Notebook (%) 

30.99 

20.90 

6.97 

ICT for looking for information 
at school (%) 




1-2 times/week 

28.74 

28.50 

27.95 

Almost every day/every day 

19.95 

19.20 

12.72 

ICT for practice/drilling at school (%) 




1-2 times/week 

15.11 

19.41 

14.8 

Almost every day/every day 

8.69 

14.18 

7.08 

ICT for doing homework at school (%) 




1-2 times/week 

13.27 

11.76 

10.52 

Almost every day/every day 

8.94 

11.13 

5.66 

At home, the student uses ICT 
for doing homework (%) 




1-2 times/week 

20.35 

23.70 

19.46 

Almost every day 

10.86 

12.75 

7.51 

Every day 

4.99 

4.18 

3.28 


4. Econometric model 

Due to space limitations, the econometric analysis is restricted the comparison 
between NP and TP. To disentangle which part of the score variation is due to the 
participation in the Program, we propose to estimate a difference-in-difference 
model. The dependent variable is the reading score of student i belonging to 
school j (Read..): 

Rea d =o+^ i +^ j +/ ear 20 , 2 +/ a r t j+/e a r 2 o l 2-Part.+£ i +fi j +v ij 
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Where X. refers to characteristics of the student and his/her family (nationality, 
age when arrived at Spain, language spoken at home, immigrant mother/father, 
lives with only one parent, minutes per week devoted to reading at home, having 
more than 100 books at home, level of education of father/mother, relation with 
economic activity of father/mother); X refers to school characteristics (size of 
municipality, class size, proportion of girls at class, proportion of immigrants 
students); Part, takes the value 1 if the Community has participated in School 2.0; 

takes the value 1 in 2012; Year ... -Part denotes the interaction between 
participation in School 2.0 and year 2012; e. and fi. denote student and school 

unobservable characteristics, and v.. is a random error term. For the estimation 

v 

of the model, the methodology proposed by OECD (2009) has been followed. 

4.1. Results for PISA (2009) and PISA (2012) 

A higher ratio of computers-per-student, as illustrated in Table 3, has a negative 
effect over reading score for non-repeaters (-75.93 points) and 2-year repeaters 
(-141.35 points). However, for the case of TP this negative effect is offset by a 
positive one (86.04 for non-repeaters, 154.87 for 2-year repeaters). 

The starting year of the Program has meaningfully influenced reading scores. It 
is negative for non-repeaters and 1-year repeaters, although smaller in absolute 
value for those who started in 2009 as compared to 2010. This could indicate that 
there is a learning curve and students need some time to come to terms with the 
new teaching methodology. On the other hand, the difference in the estimated 
coefficients between non-repeaters and 1-year repeaters is thought-provoking. 
It could be that new teaching methodologies have involved a step backward for 
1-year repeater students. 

GDP 3 per capita has been introduced as a proxy of regional purchasing power. 
The interaction with participation in the Program is positive and significant, 
although with a very small magnitude. Therefore, the results of the Program 
School 2.0 have not been conditioned by regional economic differences. 


3. Gross Domestic Product 
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Table 3. Difference-in-difference regression for reading scores 



No repeater 

1-year repeater 

2-year repeater 


Coef 

t 

Coef 

t 

Coef 

t 

C omputers-per-student 

-75.93 

-3.40 

-63.98 

-1.24 

-141.35 

-4.03 

Growth rate of computers- 
per-student 2007-2012 

0.99 

2.23 

1.04 

1.35 

1.74 

2.62 

Has notebook/digital 
pad in school 

-5.28 

-2.55 

-11.12 

-3.41 

-13.40 

-3.01 

Participation in School 2.0 

9.35 

1.36 

0.53 

0.03 

15.05 

1.27 

Year 2012 

17.64 

2.49 

32.68 

3.28 

63.62 

5.54 

Interaction with participation 
in School 2.0: 







C omputers-per-student 

86.04 

3.52 

81.55 

1.52 

154.87 

4.46 

Notebook at school 

-2.85 

-0.95 

4.90 

1.13 

4.05 

0.56 

Year 2012 

-14.35 

-1.36 

-15.46 

-0.92 

-54.51 

-2.23 

Growth rate computers- 
per-student 2009-2012 

-1.18 

-2.48 

-1.24 

-1.53 

-1.78 

-2.53 

Program started in 2009 

-14.90 

-3.33 

8.84 

0.60 

-59.68 

-3.30 

Program started in 2010 

-19.56 

-5.07 

-2.65 

-0.17 

-29.67 

-2.41 

Constant 

428.22 

56.56 

383.59 

30.11 

280.06 

19.69 

N 

14,200 


6,102 


1,762 


R 2 

0.1558 


0.1306 


0.2140 



4.2. Results for PISA-ERA (2009) and PISA-CBA (2012) 

Using the special modules of ERA (PISA, 2009) and CBA (PISA, 2012), the 
difference-in-difference model has been estimated to determine the influence 
of the Program School 2.0 over the development of digital competences (see 
Table 4). Explanatory variables are the same as in Table 3. 

The variable year 2012 is significant and negative for non-repeaters (-90.23 
points) and for 2-year repeaters (-151.41 points). This variable affects both PP 
and NP, and may gather a group of sociological determinants that have damaged 
the intrinsic value of education and learning. For the same group of students, the 
participation in the Program School 2.0 has implied an additional decrease of 
reading scores (-58.76 and -124.82 points, respectively). 
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Table 4. Difference-in-difference regression for electronic reading scores 



No repeater 

1-year repeater 

2-year repeater 


Coef 

t 

Coef 

t 

Coef 

t 

C omputers-per-student 

0.00 

-0.04 

0.01 

0.60 

-0.01 

-0.05 

Growth rate of computers 

-0.33 

-0.96 

-0.96 

-2.74 

-0.01 

-0.13 

Notebook at school 

-23.36 

-1.72 

3.59 

0.18 

-27.77 

-1.24 

Uses ICT for homework 







1-2 times/month 

7.20 

0.48 

21.83 

1.35 

-12.91 

-0.88 

1-2 times/week 

55.62 

4.66 

-0.19 

-0.01 

-54.68 

-3.39 

Almost all days 

5.67 

0.36 

0.15 

0.01 

-20.41 

-1.08 

Participation in School 2.0 

-58.76 

-2.45 

-15.45 

-0.59 

-124.82 

-4.14 

Year 2012 

-90.23 

-4.51 

24.37 

-1.06 

-151.41 

-3.19 

Interaction with School 2.0 







C omputers-per-student 

21.55 

1.02 

1.54 

0.12 

13.42 

0.72 

Notebook at school 

33.78 

1.78 

-30.54 

-1.63 

35.77 

1.58 

Year 2012 

81.00 

1.72 

-6.67 

-0.20 

136.23 

1.57 

Growth rate computers 

0.36 

0.67 

0.86 

2.12 

0.27 

0.26 

ICT for homework 







1-2 times/month 

33.74 

2.48 

24.65 

1.37 

31.512 

1.43 

1-2 times/week 

-14.20 

-1.19 

7.79 

0.37 

104.77 

5.14 

Almost all days 

34.84 

1.65 

19.14 

0.85 

56.33 

1.71 

Constant 

505.76 

13.66 

414.98 

10.23 

535.49 

9.68 

N 

4,933 


1,609 


499 


R 2 

0.2700 


0.3092 


0.6334 



Using ICT for doing homework is only significant for the category 1-2 times/ 
week for non-repeaters (55.62 points) and 2-year repeaters (-54.68 points). The 
interaction between participation in School 2.0 and ICT for homework 1-2 times/ 
week is positive and significant for 2-year repeater students (+104.77). This 
result implies that, for this specific group, there has been a remarkable difference 
in the benefit derived from the use of ICT at home between NP and PP. 


5. Conclusions 


Our results show that the increase in the provision of computers has different 
effects over reading scores based on the teaching methodology applied. The 
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increase in the provision of computers in total participant Communities leads to 
positive (although small) effects over academic performance. For TP and NP, the 
negative effect of the variable year 2012 is quite alarming. We should analyse 
which combination of factors has damaged reading scores (i.e. implication of 
families in children’s education, influence of depressive economic contexts...). 

Regarding the use of ICT at home, a moderate use (1-2 times/week) has positive 
effects for non-repeater students, although a negative one for 2-year repeater 
students. However, the interaction of participation and ICT for homework 1-2 
times/week shows a positive and significant effect for 2-year repeater students, 
which offsets the previous negative one. The implications of these results are 
twofold: (1) intensive use of ICT at home (almost every day or every day) does 
not affect academic results, but positive results emerge when they are used as a 
complement tool; and (2) the appropriate use of ICT (guided by specific teaching 
contents) may be stimulating for repeater students and help them to improve 
their academic performance. 

Regarding previous literature that ascertained a positive impact of ICT over 
student assessment, two different explanations are offered to explain the 
divergence of results. On one hand, ICT should be considered as an additional 
‘input’ in the student’s learning function, because the student can obtain more 
information and access more easily to learning resources at school and at 
home (and at any moment). On the other hand, the benefits derived of ICT are 
conditioned by the ability of the centers to modify their teaching methods, so 
that teachers and ICT become complementary. The availability of data including 
future cohorts of students participating in School 2.0 will dig into the relationship 
of ICT and student performance in Spain. 
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